Cargo cult prompting

How much of the prompt engineering is Cargo cult?

The result of a prompting depends on many factors: the version of the model (cf. new GPT allows reproducible results), randomness (temperature), and the prompt itself, including the interactive history of the “conversations”. If a prompt changes in multiple places, especially in the chat setting, it is nontrivial to attribute the change in the results to a specific change in the prompt.